Snooze: A Scalable, Fault-Tolerant and Distributed Consolidation Manager for Large-Scale Clusters

International audienceIntelligent workload consolidation and dynamic cluster adaptation offer a great opportunity for energy savings in current large-scale clusters. Because of the heterogeneous nature of these environments, scalable, fault-tolerant and distributed consolidation managers are necessary in order to efficiently manage their workload and thus conserve energy and reduce the operating costs. However, most of the consolidation managers available nowadays do not fulfill these requirements. Hence, they are mostly centralized and solely designed to be operated in virtualized environments. In this work, we present the architecture of a novel scalable, fault-tolerant and distributed consolidation manager called Snooze that is able to dynamically consolidate the workload of a software and hardware heterogeneous large-scale cluster composed out of resources using the virtualization and Single System Image (SSI) technologies. Therefore, a common cluster monitoring and management API is introduced, which provides a uniform and transparent access to the features of the underlying platforms. Our architecture is open to support any future technologies and can be easily extended with monitoring metrics and algorithms. Finally, a comprehensive use case study demonstrates the feasibility of our approach to manage the energy consumption of a large-scale cluster

Energy-Aware Ant Colony Based Workload Placement in Clouds

With cloud computing becoming ubiquitous, cloud providers are starting to deploy increasing numbers of energy hungry data centers. Energy conservation then becomes essential, in order to decrease operation costs and increase the system reliability. One traditional approach to conserve energy in these environments is to perform workload (i.e., VM) consolidation. Thereby, workload is packed on the least number of physical machines in order to increase the resource utilization and thus be able to transition parts of the resources into a lower power state. However, most of the workload consolidation approaches applied until now are limited to a single resource (e.g., CPU) and rely on relatively simple greedy algorithms such as First-Fit Decreasing (FFD), which perform resource-dissipative workload placement. In this work, we model the workload placement problem as an instance of the multi-dimensional bin-packing (MDBP) problem and design a novel, nature-inspired algorithm based on the Ant Colony Optimization (ACO) meta-heuristic to compute the placement dynamically, according to the current load. We evaluate the ACO-based approach by comparing it with one frequently applied greedy algorithm (i.e., FFD). Our simulation results demonstrate that ACO outperforms the evaluated greedy approach as it achieves superior energy gains through better server utilization and requires less machines.Avec le succès des services Cloud, les fournisseurs de ces services déploient de plus en plus de centres de données gourmands en énergie. Pour réduire les coûts et augmenter la fiabilité du système, économiser l'énergie devient essentiel. Une approche courante pour économiser de l'énergie dans ces environnements consiste à grouper les charges de travail (c'est-à-dire à grouper les machines virtuelles). Ainsi, la charge de travail est regroupée sur le plus petit nombre de machines physiques possible pour maximiser l'usage de ce sous-ensemble des ressources, et pouvoir ainsi mettre les autres ressources qui sont sous-utilisées en mode d'économie d'énergie. Cependant, jusqu'à présent, la plupart des approches fondées sur le regroupement des charges de travail se limitent à la prise en compte d'un seul type de ressource (par exemple, le processeur) et reposent sur des algorithmes gloutons relativement simples tel que le First-Fit Decreasing (FDD), qui gaspillent les ressources. Dans cet article, nous modélisons le problème du placement de charges de travail en tant qu'une instance du problème de bin-packing multi-dimensionnel, et nous construisons un nouvel algorithme bio-inspiré utilisant une méta heuristique d'optimisation inspirée des colonies de fourmis (Ant Colony Optimization, ACO) qui calcule les placements dynamiquement en fonction de la charge courante. Nous évaluons l'algorithme ACO en le comparant à l'algorithme glouton traditionnel (l'algorithme FDD). Les résultats de simulation montrent que l'algorithme ACO surpasse l'approche gloutonne en améliorant le gain d'énergie par une meilleure utilisation des serveurs et en exigeant moins de machines

Automated Risk Analysis of a Vulnerability Disclosure Using Active Learning

International audienceExhaustively listing the software and hardware components of an information system is non-trivial. This makes it even harder to analyze the risk created by a vulnerability disclosure in the context of a specific information system. Instead of basing the risk analysis of a newly disclosed vulnerability on a possibly obsolete list of components, we focus on the security team members tasked with protecting the information system, by studying how Chief Information Security Officers (CISOs) and their subordinates actually react to vulnerability disclosures. We propose to use active learning to extract the conscious and unconscious knowledge of an information system's security team in order to automate the risk analysis of a newly disclosed vulnerability for a specific information system to be defended

Including Security Monitoring in Cloud SLA

International audienceOne of the risks of moving to a public cloud is losing full control of the information system infrastructure. The service provider will be in charge of monitoring the actual infrastructure and provide the required service to clients. In our work, we aim to allow providers to provide customers with guarantees on security monitoring of their outsourced information system

Définition de SLAs pour la supervision de la sécurité dans les clouds de type IaaS : exemple d’un IDS réseau

In an IaaS cloud the physical infrastructure is controlled by service providers,including its security monitoring aspect. Clients hosting their information system need to trustand rely on what the providers claim. At the same time providers try to give assurance for someaspects of the infrastructure (e.g. availability) through service level agreements (SLAs). We aimat extending SLAs to include security monitoring terms. In our previous study [1] we proposeda verification method for security monitoring SLAs describing the performance on an networkintrusion detection system (NIDS). In this paper we address the problem of security monitoringSLA definition, specifically for the case of NIDSs in cloud. We present the following contributions.First we propose a security monitoring service description with relevant key performance indicators(KPIs). Second we propose an extension to an SLA language called CSLA [2], in order to have astandard method to define security monitoring SLAs. Third the KPIs used to describe performanceof NIDS take a base rate parameter, representing the rate of attacks in the monitored networktraffic. However, the value of the base rate is unknown at the time of SLA definition. In orderto address this contradiction, we propose a model building method and the model is used in theSLA definition. The model is used to estimate the expected performance depending on the baserate. Fourth, since there is a large number of vulnerabilities among all software products possiblyused by tenants, defining an SLA requires lots of performance evaluation tests, which makes theprocess impractical. To address this we propose a method based on rules clustering which buildsa knowledge base for NIDS performance for a large number of vulnerabilities. Finally, we presentexperiments showing the feasibility of our methods on performance estimation and clustering ofNIDS rules. We also present analysis on the shortcomings of the proposed method.Dans un cloud de type IaaS, l’infrastructure physique est contrôlée par les fournisseursde services, y compris sur l’aspect surpervision de la sécurité. Les clients hébergeant leursystème d’information doivent se fier à ce que les fournisseurs affirment. Dans le même temps,les fournisseurs essaient de donner une assurance sur certains aspects de l’infrastructure (parexemple la disponibilité) par le biais de contrats de niveau de service (Service-Level Agreementou SLA). Notre objectif est d’étendre les contrats de niveau de service afin d’y inclure des aspectsde supervision de la sécurité. Dans notre étude précédente [1], nous avons proposé une méthodede vérification du respect d’objectifs de supervision de la sécurité dans les SLAs, ces objectifsdécrivant la performance d’un système de détection d’intrusion dans le réseau (NIDS). Dans leprésent document, nous abordons le problème de la définition des SLAs portant sur la supervisionde la sécurité, en particulier dans le cas des NIDS dans les clouds. Nous présentons les contributionssuivantes. Tout d’abord, nous proposons une description du service de supervision de lasécurité avec des indicateurs clés de performance (Key Performance Indicators ou KPIs) pertinents.Deuxièmement, nous proposons une extension d’un langage de SLA appelé CSLA [2], afind’avoir une méthode standard pour définir les SLA de supervision de sécurité. Troisièmement,les KPIs utilisés pour décrire la performance des NIDS prennent en paramètre le taux d’attaquesdans le trafic réseau surveillé. Toutefois, la valeur du taux d’attaques est inconnue au momentde la définition d’un SLA. Afin de résoudre cette contradiction, nous proposons une méthode deconstruction d’un modèle et le modèle est utilisé dans la définition du SLA. Le modèle permetd’estimer la performance attendue en fonction du taux d’attaques. Quatrièmement, comme ilexiste un grand nombre de vulnérabilités parmi tous les produits logiciels éventuellement utiliséspar les utilisateurs du cloud, la définition d’un SLA nécessite de nombreux tests d’évaluationdes performances, ce qui rend le processus difficilement applicable. Pour remédier à cela, nousproposons une méthode fondée sur le regroupement de règles qui permet de construire une basede connaissances sur la performance des NIDS pour un grand nombre de vulnérabilités. Enfin,nous présentons des expériences démontrant la faisabilité de nos méthodes d’estimation des performanceset de regroupement des règles de NIDS. Nous présentons également une analyse deslimitations de la méthode proposée

Towards Self Adaptable Security Monitoring in IaaS Clouds

International audienceTraditional intrusion detection systems are not adaptive enough to cope with the dynamic characteristics of cloud-hosted virtual infrastructures. This makes them unable to address new cloud-oriented security issues. In this paper we introduce SAIDS, a self-adaptable intrusion detection system tailored for cloud environments. SAIDS is designed to re-configure its components based on environmental changes. A prototype of SAIDS is described

A New Approach to Configurable Dynamic Scheduling in Clusters based on Single System Image Technologies

Clusters are now considered as an alternative to parallel machines to execute workloads made up of sequential and/or parallel applications. For efficient application execution on clusters, dynamic global process scheduling is of prime importance. Different dynamic scheduling policies that have been studied for distributed systems or parallel machines may be used in clusters. The choice of a particular policy depends on the kind of workload to be executed. In a cluster, it is thus highly desirable to implement a configurable global scheduler to be able to adapt the dynamic scheduling policy to the workload characteristics, to take benefit of all cluster resources and tocope with node shutdown and reboot. In this paper, we present the architecture of the global scheduler and the process management mechanisms of Kerrighed, a single system image operating system designed for high performance computing on clusters. Kerrighed provides a development framework allowing to easily implement dynamic scheduling policies without kernel modification. In Kerrighed, the global scheduling policy can be dynamically changed while applications execute on the cluster. Kerrighed's process management mechanisms allow to easily deploy parallelapplications in the cluster and to efficiently migrate or checkpoint processes, including processes sharing memory. Kerrighed has been implemented as a set of modules extending Linux kernel. Preliminary performance results are presented

Snooze: A Scalable, Fault-Tolerant and Distributed Consolidation Manager for Large-Scale Clusters

Intelligent workload consolidation and dynamic cluster adaptation offer a great opportunity for energy savings in current large-scale clusters. Because of the heterogeneous nature of these environments, scalable, fault-tolerant and distributed consolidation managers are necessary in order to efficiently manage their workload and thus conserve energy and reduce the operating costs. However, most of the consolidation managers available nowadays do not fulfill these requirements. Hence, they are mostly centralized and solely designed to be operated in virtualized environments. In this work, we present the architecture of a novel scalable, fault-tolerant and distributed consolidation manager called Snooze that is able to dynamically consolidate the workload of a software and hardware heterogeneous large-scale cluster composed out of resources using the virtualization and Single System Image (SSI) technologies. Therefore, a common cluster monitoring and management API is introduced, which provides a uniform and transparent access to the features of the underlying platforms. Our architecture is open to support any future technologies and can be easily extended with monitoring metrics and algorithms. Finally, a comprehensive use case study demonstrates the feasibility of our approach to manage the energy consumption of a large-scale cluster.Une consolidation intelligente des charges applicatives et une adaptation dynamique des grappes de calculateurs offrent des opportunités importantes d'économiser l'énergie dans les grappes de calculateurs actuelles. Étant donnée la nature hétérogène de ces environnements, il est nécessaire de fournir des gestionnaires de consolidation passant à l'échelle, tolérants aux fautes, et distribués, afin de gérer efficacement les charges applicatives de ces grappes et ainsi économiser l'énergie et réduire les coûts opérationnels. Cependant, la plupart des gestionnaires de consolidation disponibles de nos jours ne satisfont pas ces critères. Ainsi, ces gestionnaires de consolidation sont pour la plupart centralisés et ne sont conçus que pour des environnements virtualisés. Dans ce travail, nous présentons l'architecture d'un nouveau gestionnaire de consolidation passant à l'échelle, tolérant aux fautes, et distribué, appelé Snooze, qui est capable de consolider dynamiquement la charge applicative d'une grappe hétérogène du point de vue logiciel comme du point de vue matériel, de grande taille, et composée de ressources utilisant les technologies de virtualisation et de système à image unique (SSI). Pour cela une API commune pour la supervision et la gestion d'une grappe est présentée. Cette API permet d'accéder de façon uniforme et transparente aux fonctionnalités des plates-formes sous-jacentes. Notre architecture est ouverte afin d'être adaptable aux technologies futures, et peut être étendue aisément avec d'autres métriques et algorithmes de supervision. Enfin, une étude complète de cas d'utilisation montre la faisabilité de notre approche pour gérer la consommation d'énergie d'une grappe de grande taille

Cloud System Evolution in the Trades (CSET): Following the Evolution of Boundary Layer Cloud Systems with the NSFNCAR GV

The Cloud System Evolution in the Trades (CSET) study was designed to describe and explain the evolution of the boundary layer aerosol, cloud, and thermodynamic structures along trajectories within the North Pacific trade winds. The study centered on seven round trips of the National Science FoundationNational Center for Atmospheric Research (NSFNCAR) Gulfstream V (GV) between Sacramento, California, and Kona, Hawaii, between 7 July and 9 August 2015. The CSET observing strategy was to sample aerosol, cloud, and boundary layer properties upwind from the transition zone over the North Pacific and to resample these areas two days later. Global Forecast System forecast trajectories were used to plan the outbound flight to Hawaii with updated forecast trajectories setting the return flight plan two days later. Two key elements of the CSET observing system were the newly developed High-Performance Instrumented Airborne Platform for Environmental Research (HIAPER) Cloud Radar (HCR) and the high-spectral-resolution lidar (HSRL). Together they provided unprecedented characterizations of aerosol, cloud, and precipitation structures that were combined with in situ measurements of aerosol, cloud, precipitation, and turbulence properties. The cloud systems sampled included solid stratocumulus infused with smoke from Canadian wildfires, mesoscale cloudprecipitation complexes, and patches of shallow cumuli in very clean environments. Ultraclean layers observed frequently near the top of the boundary layer were often associated with shallow, optically thin, layered veil clouds. The extensive aerosol, cloud, drizzle, and boundary layer sampling made over open areas of the northeast Pacific along 2-day trajectories during CSET will be an invaluable resource for modeling studies of boundary layer cloud system evolution and its governing physical processes

Vigne : vers un système d'exploitation auto-réparant pour la grille

National audienc